Papers by Ixak Sarasua Antero
DIPLomA: Efficient Adaptation of Instructed LLMs to Low-Resource Languages via Post-Training Delta Merging (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using delta-based weight merging, open-weight instruction-tuned large language models can be efficiently adapted to low-resource languages without costly large-scale post-training. |
| Approach: | They propose a delta-based instruction-preserving language adaptation strategy that decouples adaptation from post-training alignment by pretraining a foundational LLM on monolingual target-language data and then injecting instruction-following capabilities via delta-derived weight merging. |
| Outcome: | The proposed method achieves significant gains in instruction-following, linguistic proficiency, and safety on Basque, Welsh, and Swahili while preserving the original model's multilingual performance. |
Pipeline Analysis for Developing Instruct LLMs in Low-Resource Languages: A Case Study on Basque (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models are typically optimized for resource-rich languages like English . however, the proprietary nature of these models makes them impractical for many researchers and developers. |
| Approach: | They propose to develop large language models that can follow instructions in Basque . they focus on three key stages: pre-training, instruction tuning, and alignment with human preferences . |
| Outcome: | The proposed models improve natural language understanding (NLU) of the foundational model by 12 points . the results show that the models can follow instructions in Basque with human preferences . |